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DETAILED ACTION 



Specification 



1 . The abstract of the disclosure is objected to because of its length. 

A brief abstract of the technical disclosure in the specification must commence on a 
separate sheet, preferably following the claims, under the heading "Abstract " or "Abstract of 
the Disclosure." The abstract in an application filed under 35 U.S.C. Ill may not exceed 150 
words in length. The purpose of the abstract is to enable the United States Patent and Trademark 
Office and the public generally to determine quickly from a cursory inspection the nature and 
gist of the technical disclosure. The abstract will not be used for interpreting the scope of the 
claims. See MPEP § 608.01(b). 

Appropriate correction is required. 

2. The disclosure is objected to because of the following informalities: inconsistencies. 

o Reference sign 53 is first described in reference to figure 5; however, it is shown in 
figures 2 and 3, both of which were described prior to figure 5. 

o On page 17, line 7 states, "signal b(x,y)"; however, figure 7 shows h(x,y). 
Appropriate correction is required. 



3. 



Claim Objections 

Claim 2 is objected to because of the following informalities: antecedent basis issue. 
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The claim language recites therein, "wherein the image sensor", however Claim 1 does 
not introduce an image sensor rather introduces "an image sensing device". For the purposes of 
examination, the Examiner will interpret Claim 2 using "wherein the image sensing device". 

Appropriate correction is required. 

4. Claim 6 is objected to because of the following informalities: indefinite claim language. 
The claim language recites therein, "wherein the slow response photosites have a 

response that is slower by at least one stop compared to the fast photosites". Claim 1 simply 
defines fast photosites as having a predetermined response to light exposure and slow photosites 
with a slower response to the same light exposure. In camera technology, a "stop" corresponds 
to the size in an aperture opening in a camera. However, neither has Claim 1 nor 6 introduced a 
relationship between the "fast" and "slow" photosites and aperture opening nor have they 
introduced a relationship between the size of the aperture opening corresponding to "one stop". 
For those reasons, Claim 6 is indefinite. For the purposes of examination, the Examiner will 
interpret Claim 6 as the following "wherein the slow response photosites have a response that is 
slower compared to the fast photosites". 
Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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6. Claims 1, 3 - 6, and 8 - 17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Merrill in view of Melen. 

7. For Claims 1, 12, and 17, Merrill disclose, as shown in figures 1 A, 6, and 7 and as stated 
in column 4 (lines 53 - 61), 10 (lines 33 - 39), and 1 1 (lines 19 - 28 and 40 - 54), an image 
capture system (see figure 7) for generating an extended dynamic range digital image, 
comprising: 

a) a image sensing device having fast photosites (Photodiode 12) with a predetermined 
response to light exposure interspersed with slow photosites (Photodiode 14) with a slower 
response to the same light exposure for producing a sparsely sampled high resolution digital 
image having fast pixel values produced by the fast photosites and slow pixel values produced by 
the slow photosites (see explanation below); 

Merrill employs an image-sensing device comprised of at least an image sensor (200; see 
figure 6) wherein the image sensor is an active pixel sensor comprised of larger photodiodes (12; 
"fast") interspersed among (14; "slow") photodiodes, wherein the image capture system uses 
both the "fast" and the "slow" photosites to expand the dynamic range. Furthermore, Merrill 
disclose, as shown in figure 7 and as stated in column 1 1 (lines 2- 35), "a block diagram of an 
illustrative digital camera employing three" image sensor arrays (200). A color-separating prism 
(232) is shown mounted inside light-tight housing (234). Attached to the three output faces of 
prism (232) are the three image sensors(246R, 246G, and 246B), aligned in such a way that the 
color-separated images fall on their active areas in registration. Each on-board assembly 
includes an imaging array of pixel sensors like that partially depicted in figure 6. 
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However, Merrill does not disclose a digital image processor for expanding the dynamic 
to yield a full resolution digital image having an extended dynamic range. Although, Merrill 
clearly states, in column 2 (lines 33 - 56), "one problem encountered with prior-art imagers is a 
limitation on the dynamic range of images that can be captured by the array. Images that contain 
both low-light-level pixels and high-light-level pixels could be improved if the dynamic range of 
the imager could be increased. In an active pixel sensor, the sensitivity of measuring charges 
generated by photons can be described as a charge-to- voltage gain or light-to-output- voltage 
transfer gain. Voltage dependence of the photodiode capacitance and other capacitances, and 
nonlinearities of the readout amplifier transistor can make the gain vary with level, so that the 
overall transfer curve may be somewhat nonlinear. A nonlinearity in which higher light 
intensities give lower gains is said to be compressive. A significant degree of compressive 
nonlinearity can have a beneficial effect on the signal-to-noise ratio of the image at low light 
levels, and can thereby enhance the usable dynamic range of the imager." Therefore, since it is 
objective of Merrill to increase the dynamic range of the imager, it would have been obvious to 
one with ordinary skill in the art to have provided a digital image processor for processing the 
outputs of the "fast" photodiodes (12) and the "slow" photodiodes (14) to yield an expanded 
dynamic range high resolution digital image. 

Lastly, Merrill does not disclose, in the very least, an encoder for reducing the dynamic 
range of the full resolution digital image to fit within the dynamic range of a storage space and a 
digital image store. 

On the other hand, Melen also disclose an image capture system for generating and 
storing an extended dynamic range digital image. More specifically, Melen teach, as shown in 
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figures 1 and 4(c) and as stated in columns 2 (lines 47 - 51), 3 (lines 9 - 49), 4 (lines 52 - 66), 
and 6 (lines 4 - 1 1), the image capture system comprising: 

c) an encoder (comprised of components 101, 1 18, 124, and 122) for reducing the 
dynamic range of the full resolution digital image to fit within the dynamic range of a storage 
space (102) having a dynamic range less than the dynamic range of the full resolution digital 
image to form a limited dynamic range digital image (120) represented in the storage space (see 
column 4, lines 52 - 56) and for producing a residual image (image signal 103, corresponding to 
image signal 110, after being output from component 101 and being input into component 124) 
representing a difference between the full resolution digital image (image signals 110 and 112) 
and the limited dynamic range digital image (120) that can be used with the limited dynamic 
range digital image to reconstruct the full resolution digital image (see column 3, lines 35 - 39); 
and 

d) a digital image store (102) for storing the limited dynamic range digital image in 
association with the residual image. 

In summary, Melen teaches, that the full resolution digital image is comprised of image 
signals 110 and 1 12 and is broken into a residual image (image signal 103 input into component 
124) and a limited dynamic range digital image (120), both of which are stored in the digital 
image store (102) for reconstruction. As stated in column 3 (lines 35 - 49), at the time the 
invention was made, one with ordinary skill in the art would have been motivated to include an 
encoder and a digital image store, as taught by Melen, in the image capture system, disclosed by 
Merrill, as a means to store a high resolution wide dynamic range digital image which can be 
accurately reconstructed for display at a later time. Therefore, at the time the invention was 
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made, it would have been obvious to one with ordinary skill in the art to have included an 
encoder and a digital image store, as taught by Melen, in the image capture system, disclosed by 
Merrill. 

As for Claims 3, 13, and 14, Merrill employs an image capture system (see figure 7) is 
comprised of three identical image sensors (200) wherein each the image sensors (200) is 
interspersed with "fast" photosites (photodiode 12) and "slow" photosites (photodiode 14), 
wherein the image capture system uses both the "fast" and the "slow" photosites to expand the 
dynamic range. 

8. As for Claim 4, Merrill shows through obviousness (see rejection of Claim 1), as shown 
in figure 7, a digital camera comprising a digital image processor for expanding the dynamic 
range of an image included in the digital camera. However, Merrill does not disclose wherein 
the image sensing device is located in a digital camera and the digital image processor is located 
in a host computer separate from the digital camera. Official Notice is taken that both the 
concepts and advantages of locating a digital image processor in a host computer separate from 
the digital camera are well known and expected in the art. It would have been obvious to one 
with ordinary skill in the art to have included a digital image processor in a host computer 
separate from the digital camera as means to perform advanced image processing such as 
photofmishing. 

9. As for Claim 5, Merrill shows through obviousness (see rejection of Claim 1), as shown 
in figure 7, a digital camera comprising a digital image processor for expanding the dynamic 
range of an image included in the digital camera. 
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10. As for Claim 6 (please see objection above), Merrill discloses, as described above, 
wherein the slow response photosites have a response that is slower compared to the fast 
photosites. 

11. As for Claim 8, Melen teaches, as shown in figure 2 and as stated in column 3 (lines 35 - 
39), an image reconstructor (see figure 2) that employs the residual image (128) and the limited 
dynamic range digital image (126) to form a reconstructed full resolution digital image. 

12. As for Claim 9, Melen teaches, as shown in figure 2, wherein the image reconstructor 
applies an image modification (resize and position component 201) to the reconstructed full 
resolution digital image. 

13. As for Claim 10, Melen teaches, as shown in figure 1 and as stated in column 3 (lines 9 - 
29), the limited dynamic range digital image (126) and the residual image (128) are stored 
individually in the digital image store (102). There is no indication or suggestion whatsoever in 
Melen as to the combination prior to storage of the two images, hence they are separate image 
files. However, Official Notice is taken that both the concepts and the advantages are storing 
two digital images in the same digital image file are well known and expected in the art. It 
would have been obvious to one with ordinary skill in the art have stored two digital images in 
the same digital image file as means to decrease the number of files stored in the storage space, 
thereby increasing the speed of image reconstruction. 

14. As for Claim 11, Melen teaches, as shown in figure 1 and as stated in column 3 (lines 9 - 
29), the limited dynamic range digital image (126) and the residual image (128) are stored 
individually in the digital image store (102). There is no indication or suggestion whatsoever in 
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Melen as to the combination prior to storage of the two images, hence they are separate image 
files. 

15. As for Claim 15, the claim language requires a residual image representing a luminance 
difference and a residual image representing a chrominance difference are produced and stored. 
This particular claim language does not require two separate residual images, rather, interpreted 
broadly, can be a single residual image comprising both luminance and chrominance differences. 
Melen does teach of a residual image comprising both luminance and chrominance differences 
(see Examiner's residual image interpretation above). 

16. As for Claim 16, the claim language requires wherein the residual image is stored as tiles 
representing subsets of pixels in separate tags in an image files. Claim 16 is simply requiring 
that the residual image be stored in a TIFF file format. Melen, however, only teaches of storing 
the residual image in JPEG file format. Official Notice is taken that both the concepts and 
advantages of storing an image in TIFF file format are well known and expected in the art. It 
would have been to obvious to one with ordinary skill in the art to have stored the residual image 
in TIFF file format, rather than JPEG file format, because TIFF file format is a loss-less 
compression format as opposed to the lossy JPEG compression format. 

17. Claims 2 and 7 are rejected under 35 U.S.C. 103(a) as being unpatentable over Merrill in 
view of Melen in further view of Bayer. 

18. As for Claim 2, Merrill employs an image-sensing device comprised of at least an image 
sensor (200; see figure 6) wherein the image sensor is an active pixel sensor comprised of larger 
photodiodes (12; "fast") interspersed among (14; "slow") photodiodes, wherein the image 
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capture system uses both the "fast" and the "slow" photosites to expand the dynamic range. 
Furthermore, Merrill disclose, as shown in figure 7 and as stated in column 1 1 (lines 2- 35), "a 
block diagram of an illustrative digital camera employing three" image sensor arrays (200). A 
color-separating prism (232) is shown mounted inside light-tight housing (234). Attached to the 
three output faces of prism (232) are the three image sensors(246R, 246G, and 246B), aligned in 
such a way that the color-separated images fall on their active areas in registration. Each on- 
board assembly includes an imaging array of pixel sensors like that partially depicted in figure 6. 

However, Merrill does not disclose a color image sensor having a color filter array and a 
color filter array interpolator for interpolating color values. 

On the other hand, Bayer also disclose an image sensing device. More specifically, 
Bayer teach, as shown in figures 1A, IB, and 4 and as stated in columns 3 (lines 51 - 68), 4 
(lines 1 and 2), and 13 (lines 13 - 30), color image sensor (32) having an array of photosites (22) 
and a color filter array (24) arranged over the array of photosites; wherein the image processor 
includes a color filter array interpolator (40) for interpolating color values at all of the photosites. 
As stated in column 2 (lines 9 - 15), at the time the invention was made, one with ordinary skill 
in the art would have been motivated to include a color image sensor having an array of 
photosites and a color filter array, as taught by Bayer, in the image capture system, taught by 
Merrill in view of Melen, as a means to reduce optical and image registration problems and to 
provide a more rugged camera structure. Therefore, at the time the invention was made, it would 
have been obvious to one with ordinary skill in the art to have included a color image sensor 
having an array of photosites and a color filter array, as taught by Bayer, in the image capture 
system, taught by Merrill in view of Melen, 
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19. As for Claim 7, Bayer teaches wherein the color filter array is a Bayer array. 



20. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. The following is a brief description of each of the prior art: 

o Prior Art C, D, E, and F disclose an apparatus and corresponding method for 
expanding the dynamic range of image signals by varying the exposure times and/or readout 
method of a single image sensors or a plurality of image sensors. 

o Prior Art G, H, N, and O disclose a image sensor comprising "fast photosites" 
interspersed with "slow photosites" wherein the "fast" and "slow" photosites are created by 
varying the size and/or shape of the microlens array disposed therein. 



Conclusion 
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Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Justin P Misleh whose telephone number is 703.305.8090. The 
Examiner can normally be reached on Monday through Thursday from 7:30 AM to 5:30 PM and 
on alternating Fridays from 7:30 AM to 4:30 PM. 

If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor, Wendy R Garber can be reached on 703.305.4929. The fax phone number for the 
organization where this application or proceeding is assigned is 703.872.9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



JPM 

June 25, 2004 




